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ABSTRACT 


Research is underway at the NASA Johnson Space Center on the development of vision 
systems that recognize objects and estimate their position by processing their images. This is a crucial 
task in many space applications such as autonomous landing on Mars sites, satellite inspection and 
repair, and docking of space shuttle and space station. Currently available algorithms and hardware 
are too slow to be suitable for these tasks. Electronic digital hardware exhibits superior performance 
in computing and control; however, they take too much time to carry out important signal processing 
operations such as Fourier transformation of image data and calculation of correlation between two 
images. Fortunately, because of the inherent parallelism, optical devices can carry out these 
operations very fast, though they are not quite suitable for computation and control type of 
operations. Hence, investigations are currently being conducted on the development of hybrid vision 
systems that utilize both optical techniques and digital processing jointly to carry out the object 
recognition tasks in real time. 

The author of this report, during his tenure as a summer faculty fellow at the Johnson Space 
Center studied the various aspects of this research. He collaborated with Dr. Richard juday and his 
colleagues on developing algorithms for the design of optimal filters for use in hybrid vision systems. 
Specifically, an algorithm was developed for the design of real-valued frequency plane correlation 
filters. Further, research was also conducted on designing correlation filters optimal in the sense of 
providing maximum signal-to-noise ratio when noise is present in the detectors in the correlation 
plane. Algorithms were developed for the design of different types of optimal filters: complex filters, 
real-valued filters, phase-only filters, ternary-valued filters, coupled filters. This report presents 
some of these algorithms in detail along with their derivations. 
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INTRODUCTION 


Recognition of objects by machines is a vital task in many applications related to NASA 
missions. Docking of space shuttle and space station, extra-vehicular activities by robots, and 
autonomous landing on Mars are some examples of such applications. To carry out the tasks 
efficiently and accurately in real time, suitable technologies are being investigated. Most of these 
tasks involve some form of processing of images acquired using a camera. A straightforward 
approach is to use electronic digital hardware to carry out the necessary processing involved in the 
recognition of objects. Researchers in many universities and research laboratories around the world 
are working on the development and evaluation of algorithms for such processing and pattern 
recognition. One major drawback of the electronic processing technique is that at the current 
technology, the algorithms take too much time to be suitable for real time applications. 

An alternate technology that is under investigation by many researchers is the use of optical 
processing. Since through optical techniques processing can be done in a parallel mode, very high 
speed can be achieved. Taking Fourier transformation, multiplication of a signal by a filter, and 
obtaining correlation function of two images are some of the operations that optical processing can 
efficiently perform. However, some operations such as storage and retrieval of data, arithmetic 
operations, etc., cannot so easily be done using optical techniques. Hence, a hybrid system employing 
optical techniques for correlation filtering and electronic digital techniques for image storage, 
retrieval, correlation plane processing, etc., is being investigated for development and use in vision 
systems. 

One major area of investigation in the development of hybrid vision systems is the design of 
optimal filters that yield high signal to noise ratio (SNR) facilitating accurate detection of objects 
in the presence of noise and other extraneous objects. During the summer fellowship period, the 
author collaborated with his JSC colleague Dr. Richard Juday and Dr. B.V.K. Vijaya Kumar of 
Carnegie Mellon University and developed algorithms for the design of several classes of optimal 
filters: complex filters, real-valued filters, ternary valued filters, phase only filters, and coupled 
filters. Some of these algorithms and their derivations will be presented in the following sections. 

CORRELATION FILTERS 

The theory of correlation filters for the recognition of known objects has been well discussed 
in the literature [1]. It basically involves the following steps. From a knowledge of the reference 
image s(x) (we will use 1-D notation for the sake of simplicity), a filter function h(x) is designed. 
When an image t(x) is to be tested to find whether the reference image is present in the test image or 
not, the image t(x) is convolved with the filter h(x) to yield c(x). If c(x) has a well pronounced peak 
above a preassigned threshold, then it is concluded that the reference image is present in the test 
image; otherwise it is not. If a peak is present, from the position of the peak, one may also estimate 
the location of the reference image in the test image. When h(x) is equal to s(-x), the matched filter 
of the reference image, it can be shown that c(x) will yield the cross correlation between the test 
image and the reference image. Hence, these filters are called correlation filters. 

The implementation of correlation filters in the space domain is quite time consuming. An 
alternate approach is to transform the test and reference images into the Fourier frequency domain, 
multiply the transforms and then perform the inverse transform to obtain the filter output c(x). If 
one uses digital techniques using electronic hardware, the Fourier transformation is also equally time 
consuming. However, Fourier transformation can easily and almost instantaneously be carried out 
using optical techniques. As a result, considerable research is being conducted on the design of optical 
correlators for real time pattern recognition purposes. Since the invention of the holographic matched 
filter for optical correlations by VanderLugt [2] in 1964, many other filters with improved 
performance have been proposed. The accuracy of detection of these correlation filters depends to 
a large extent on the absence of noise in an image. It may be noted that the nonreference image 
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components present in an image may also be considered as noise. To account for the presence of noise 
and to measure the performance of a filter in the presence of noise, signal-to-noise ratio (SNR) has 
been introduced and procedures have been devised to design optimal filters that maximize the SNR 
and thus improve the accuracy of identification. 

Derivation of maximum SNR filters of different types (complex, phase only, ternary and 
coupled filters) in the presence of image-plane noise but no detector noise has been presented in 
literature. For the case of real filters, derivation of optimal filters starting from the fundamental 
expression for SNR has not been done so far. We first present such a derivation obtained during the 
fellowship period. This result has been submitted for publication [3]. 

OPTIMAL REAL CORRELATION FILTERS 


Let s(x) denote the reference image and S(f) its Fourier transform. Let H(f) denote the filter 
transform. In the absence of input noise the resulting correlator output at the origin is given by 

c( 0) - J S(f)H(f)df W 


where the limits of integration are those implied by the bandwidths of S(f) and H(f) (whichever has 
a smaller bandwidth.) A model for possible uncertainties in the input is the additive noise n(x). We 
model this as a sample realization with mean and power spectral density P n (f). The additive noise 
n(x) in the input leads to randomness in the output c(0). We can show that 

£{c(0)} - 0) h- J S(f)H(f)df (2) 


and 


VarMO)) - JP n (/) !//(/) IV/ 


( 3 ) 


When the input to the correlator is only noise n(x), the output c(0) will be a random variable with 
mean H(0) and variance as given in Eqn. (3). For good detection, we need to separate the two 
means as much as possible while keeping the variance small. A convenient measure for this is the 
signal-to-noise ratio (SNR) defined as below: 


SNR - 'J ( 4 ) 

We will next derive the optimum real filter H(-) that maximizes the SNR. The derivation can be done 
in more than one way. In the following we use discrete arguments and partial differential equations. 
For the continuous argument case one may employ variational calculus and obtain identical results. 
One may also employ Cauchy-Schwartz inequality to obtain the same result. We use such a technique 
in the next section where detector noise is taken into account. 


21-4 



Let us sample the frequency domain quantities at intervals A / to produce amplitudes A k and 
phases * k : 


/4 k exp( y> k ) - S(kAf) 

H k - H(kAf) (5) 

^nk - W*/) 


Then 


SNR - 


t 


524// k exp(-y> k ) 

k 


E"fck 


m 


where l and k are summed over values appropriate to the filter bandwidth. For an extremum SNR, 
each filter value H m is chosen so that the SNR is stationary; i.e., subject to the constraint 

JL(SNR)-0 . (7) 

dr/ 

m 


Defining p and B real > 0 such that 

B expO'jS ) - 53 >4 k // k exp(y# k ) (8) 

k 


and taking the partial derivative in (7) we get 




k 


Bcos( 4 >-fi) 


(9) 


We have, then, 


H 



cos(* m -0) 


( 10 ) 


with proportionality constant independent of m. The SNR is independent of a constant multiplier of 
H k , so we may as well make Eqn. (10) an equality. Then we need only solve for p. Eqn.(10) indicates 
several interesting things. As might have been expected, the optimum-SNR real filter value is 
directly proportional to the local amplitude of the reference signal transform and inversely 
proportional to the local noise power. The novelty in this result is that the filter is also weighted 
according to how well the signal’s local phase ($ m ) lines up with the phase (p) of the filtered result. 

Now, to determine the filter completely, we need to find the value of p. For this we need to 
solve the nonlinear equations 
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cos(# m - P ) 


( 11 ) 


where 





P 


Arg 


^// k ^ k exp(> k )J 


A traditional method of solving these equations is as follows: Choose a trial value for p = p v Solve 
for H m and see if the result is consistent. That is, if the p calculated using the second expression is 
equal to p v then we have a stationary SNR. This is similar to the previous [4] approaches where 
calculating a filter giving an extremum in correlation intensity involved a search over a single 
parameter. Relating to the present case, we would require a search over p. As a significant advance, 
we have been able to separate p in Eqn. (11). The search for p is over for real filters. Substituting 
for H m in the expression for p in Eqn. (1 1) we have 


p - Arg 


£ j!lLcos(* k -0 )y4 k exp( > k ) 

k Mik 


- Arg 


£ -^-[expOV k )exp( -jp ) +exp( -j* k )exp(Jp )]-4 k expO> k ) 

k ' nk 


( 12 ) 


Arg 


expO> ) 


A 2 ) a 2 

£ _LexpO'20 k ) |exp( -j2p )+J^ -J- 

k * nk J k ' nk 


The above will be true if and only if 


P ~ (-j)Arg 


E 

k 


(A k exp OV k )) J 


nk 


+ nl; where n is an integer 


(13) 


By substituting for p in the expression for SNR, we can show that n should be equal to zero for 
maximum SNR. 

We now give an abbreviated discussion of the derivation of the optimal real filter in the 
continuous domain. Define u and v as follows. 

a - {j//(/M(/)exp[ +>(/)]<//} {imf)A(f)exp[-Mf))df\ 

v-J H\f)P n (f)df (14) 

SNR- — 
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(The region of integration, ordinarily a single diffraction order, is implicit.) Following the variational 
calculus approach given in more detail elsewhere [4], we denote the arbitrary disturbance function 
as n ( •), so that the deviated value of the optimum real filtering function is H(-) + <«/*(■) for a 
(presumed small) scalar, a. The deviated value of the correlation field is obtained by replacing H(-) 
by H(-) + m(-). Let u a be u evaluated with H(-) + <m(‘) replacing H(-) and similarly for v a . The 
first-order variation in SNR induced by an(') is 


i(SNR) - a 


d_ 


a-0 


That a given function H(-) produces an extremum of SNR is stated as 


(15) 


£(SNR)-0 , implying v su-u Sv . ' 

A relatively straightforward extension of the method shown earlier leads to a form very similar to the 
discrete case. 


(I7) 

e 2 - ArgJtf(/H(/)exp[j*(/))<f/ 

We have consistency if p^ = p 2 - These expressions are of the same form as those obtained for the 
discrete case with the replacement of integration in place of summation. Then applying the same 
development as in Eqns. (12) - (13), we get 


P - (-i)Arg [ 

- (|)Arg f 


rjJ) j 

[wf-iwr 

W 


(18) 


df <■ j2 f 5r</) ^' (/) <(/| 
' 1 > />„(/) 


We will now consider some special cases . The discussion in the following is restricted to the 
continuous case. Results for the discrete case are exactly analogous, as is apparent from the 
congruence between Eqns. (11) and (17). 


i) s(x) is real. 


Then 


. S K (f) 5,(/) 

j w 


df 


- 0 


- 0 or 


T 

T 


(19) 


ii) s(x) is 


real and 


J PJJ) 


df ~ J 


df. Then p is arbitrary. 

w 


(20) 


21-7 



iii) s(x) is complex and J df - J df. Then p - 1 (21) 

Cases i) and ii) correspond to the cases considered by Kumar [5] and case iii) is a new result. 

The SNR of the real filter, obtained by substituting the optimal value of H(f) and simplifying 
the expression, is given by 


S^RMF 


lr 

2 J P n (f) 



Mf) 


df\ 


( 22 ) 


where SNR^p represents the SNR of the real matched filter. A relative measure of performance 
of the real filter in comparison with the complex matched filter is obtained as 

i f Bntdf i 

SNRrmf _ + J P n (f) (23) 

SNRcmf 2 2 r | SU)f j, 

J PnU) 

where SNR CMF represents the SNR of the complex matched filter. In the above expression we see 
that the SNR of the optimal real filter is less than or equal to the optimal complex filter. However, 
the maximum loss that results from the use of our optimal real filter is 3 dB. 

Figure 1 in the earlier paper [5] shows the loss in SNR (relative to the matched filter) when 
using the optimal real filter. That is valid here also. Only what is meant by "normalized even-part 
energy" changes depending on P n (/). As shown before, we lose at most 3 dB in SNR (compared with 
the matched filter) when we use the optimum real filter. This discussion also shows that the optimal 
real filter is not the amplitude part of the complex matched filter. 

DETECTOR NOISE AND SNR 

When the correlation output c(0) is detected by a photodetector, several things happen. 
Detectors respond only to |c(0)|, thus ignoring all phase information. Also, the detectors introduce 
a gain and some noise. An accurate model for detector noise is complicated and must include the 
signal dependent nature of detector noise. Instead, we will use the following simple model for y, the 
detector output: 

y - c( 0) ♦ « d . ( 24 > 


In this detector noise model, we assume (without loss of generality) that the detector gain is unity and 
that the detector noise n d is additive. The noise n d is assumed to have a mean n d and variance a d 2 . 
The additive assumption is somewhat questionable. However, it makes the analysis tractable and it 
helps us to illustrate the main point we want to emphasize (i.e., we must trade-off input noise 
tolerance for detector noise tolerance). We will now find the mean and variance of y in Eqn. (24) 
for the two possible input cases. 
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When the input contains only noise n(x) let the output be y 0 . Then the mean and variance of 
y 0 are given as follows: 


E{y 0 ) - + "n 


(25) 


and 


Var[y 0 ) - c\ ♦ J />„(/)!//(/) I 2 df 


(26) 


When the input contains signal s(x) corrupted by additive noise n(x), let the output be y^ Then the 
mean and variance of y 1 are given as below: 


E{y i) - "d + + J S(J)H{J)df 


(27) 


and 

Va « yi ) - o\ ♦ jP n (f)\H(f)\ 2 df 


(28) 


Using the statistics in Eqns. (25)-(28), the SNR in the presence of detector noise can be expressed as 

SNR a. 

jtVariyJ + Var(y 0 )) (M) 

l[5(/)//(/)rf/1 2 

4 + jpjfmniw 

Note that the only difference between the SNR expressions in Eqn. (4) and Eqn. (29) is the extra o d z 
in the denominator of Eqn. (29). However this makes the optimal filter choices for the two SNR’s 
different. When is very small (compared to the input noise term), the two SNRs are identical and 
the previous optimal filters will still be optimal. However, when ^ is very large ( such that the input 
noise term can be ignored) SNR is simply proportional to |£(c(0}| 2 and we must simply maximize the 
correlation value at the center. In the next section we derive expressions for H(f) that maximize the 
SNR in Eqn. (29). 

OPTIMAL FILTERS WHEN DETECTOR NOISE IS PRESENT 

In this section we will determine H(f) that maximize the SNR in Eqn. (29). There are five 
different cases: (1) Complex filters, (2) real filters, (3) phase only filters (4) binary phase only filters 
and (5) coupled filters. In each case while determining these filters, we must use the condition 

!//(/)!<! (30) 


l£{j' 1 }-£{j' 0 >l 2 
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since the transmittance of an optical filter can never exceed unity. Also, without this constraint, the 
detector noise variance <r d 2 will not be meaningful. 

Complex Filters 

Let us allow H(f) to be complex. Let the filter energy E h be defined as follows: 

E h - \\HU)\ 2 df < 31 > 


Then the SNR in Eqn. (29) can be written as below: 

SNR - 1 5(/)//(/)rf/ ' 2 

j[*- d - p n (f)mn\ 2 df 


(32) 


where 



(33) 


is the variance of an equivalent white noise at the input which gives the same effect at the origin of 
the correlation plane as the detector noise with variance <r d 2 for the given filter H(f). 

To find the optimal choice of H(f) we now apply the Cauchy-Schwartz inequality to the 
numerator to get the following: 


\jS(/)H(f)df\ 2 - IJ 


J 


S(f) 


+ pJf) 

\S(f )\ 2 


hu) + />„(/) 


df \ 2 


df 


r«, j 


(fl//(/)l 2 [4 - PJJ)]df\ 


(34) 


Substituting Eqn. (34) in Eqn. (32), we obtain the following result: 

SNR s [ d! - SN/i m „(4) (35) 

4 * rjJ) 


with equality occurring if and only if 


HU) - <* 


S m U) 

4 * w 


(36) 
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where a is chosen to satisfy the constraint that the maximum magnitude of the filter is unity. Then 
the optimal filter H(f) can be written as 


*opt</> - 


+ P n U) 1 

\S\f)/l°l ♦ WJUax 


(37) 


As a jd 2 is related to the detector noise <7 d 2 , this filter will also be optimal for a detector noise <r d 2 = 
a id 2 E h where 


E h - j\H(/)\ 2 df 


(38) 


Thus starting with <r id 2 , we can design an optimal filter and find the corresponding detector noise for 
which this filter is the optimal one. To design a filter for a given o d 2 , we may use either an 
optimization scheme or do the following: Obtain a graph relating c id 2 and o d by designing a number 
of filters with different values of a ld 2 . Then from the graph, a suitable a xd is picked for a given a d . 
Once we know -id 2 , we can design the optimal filter. It may be verified that for the special case of 
P n (f) = a constant, the classical matched filter is still the optimal filter. 

Real Filters 


Let us now consider the case when H(f) is real. Then using the same notation for E h as above, we 
have 


1 5(/) " (/W|2 

o 2 d + \P n {f)\H{f)\ 2 df 


\\SU)H{f)df\ 2 
2 


f(^ + p n (/))i//(/)i 2 d/ 

J E h 


CC * 

J/ > nt (/)itf(/)i 2 rf/ 


(39) 


where 


c - jS(/)mf)df, and 



Expressing cc* as 


cc* 


/ 


\2 


CC* + c*c 

k 2131 


we get 


SNR - 


2lcl ; J 

lP at (f)M 2 df 


(40) 


(41) 


(42) 
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As for real filters H* = H, we can write the expression for SNR as 


SNR 


r (cS'(f)+c*S(/)) H y) d f 

{) 21c I J 


JP nt (/)|//(/)| 2 rf/ 


( 43 ) 


Applying Cauchy-Schwartz inequality we get 


SNR 


, cS-(f ) + c-S(f )^ t(f) H{f)df 

[ J 2\e\JpJfT 


jP nt (/)|//(/)| 2 rf/ 

Equality (and hence maximum SNR) is achieved iff 




cS*(f)+c*S(J) 


2\c\JpJn 


df 


CSV) * c'S(f) _ . IS(/)I „ 

H(f) ‘ 2id pjn ■ - ° Tjj) { fi) 


(44) 


(45) 


where # # (f) = /4rg{S(f)}, p = Arg{c) and <* is a real constant such that the maximum response condition 
is satisfied. To determine p, substitute the optimal H(f) in the expression for c (Eqn. (40)) and 
simplify as 


\c\e ifi - 1 
2 


\' 3Ndi . 

> p « u ) ' Ktn 




(46) 


The above condition will be satisfied if and only if 


P 


Arg\ [S ^ 2 df + m 

J PmV) 


(47) 


where n is an integer. Further we can show that SNR will be larger if n = 0 (or any even integer) than 
when n is odd. Thus p value is given by 


P 



[SV)V 

(/) 


df 


(48) 


With this p value the maximum SNR value can be shown 

SNR m . 


to 


irMlli/ *1 

2 1 PJf) 2 


be 

• [S(/)l 2 


dA 


nt 


(49) 
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The optimal H(f) satisfying the maximum condition is given by 


*opt(/> - 


l^icosw/)-^ 

r ntv > 


( 50 ) 


The above H(f) is optimum for a given input noise. To determine H(f) for a given detector noise one 
can follow a procedure similar to that used in the complex filter case of either using an optimization 
scheme or setting up a graph between a id 2 and a d 2 and selecting a suitable a id 2 for a given a d 2 . 

Phase Only Filter 


Let H(f) be a phase-only filter (POF) with a region of support R defined below: 


//(/) ' 


0 


for f € R 

for f € R 


(51) 


Substituting this in Eqn. (29) 


we obtain the following expression for SNR: 

I2 


SNR - 


|IS(/)le j[ *’ (f) ** h( °W 


T- 


U)df 


(52) 


The denominator in Eqn. (52) does not depend on $ h (f) and the numerator is maximized by choosing 
^(f) = -$ g (f), i.e., the conventional POF maximizes the SNR in the presence of detector noise. The 
next task is to find the optimal region of support R. Let A R denote the area of the region R. Then 
the SNR in Eqn. (52) can be rewritten as follows: 


SNR - 


( ^ 
|LS(/)lrf/ 


I 


w 


df 


(53) 


The SNR expression in Eqn. (53) is identical to the SNR expression derived elsewhere [6] for the case 
of no detector noise and input colored noise with spectral density [P„(f) + o 2 / A R }. It is shown there 
that the region of support R that maximizes the SNR must be of the form: 



IS(/)I 

>r 

/ \ 


2 



[ a r j 



( 54 ) 
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where T is an unknown threshold. Since A R depends on R , Eqn. (54) is an implicit equation. An 
algorithmic procedure for determining R is given below: 

To illustrate the workings of the algorithm, let us assume that S(f) is represented by an array 
of 64 X 64 = 4096 pixels. Each frequency in this array may or may not be included in R. Thus there 
are 2 4096 possible choices of R from which we must select the one that maximizes the SNR. 
Exhaustive search is obviously impossible, but the characterization in Eqn. (54) proves useful. For 
example, there are 4096 different choices for R with A R = 1. Among all of them, the best R is the 
one that includes the highest value of |S(f)|/[a d 2 + P n (f)]. Extending this to other A R values, we get 
the following algorithm: 

Step 0: Start with A R = 1 
Step 1: Compute 


Step 2: Arrange sampled values of G(f) in the descending order as Gj > G 2 > ... > G N > 0. 

Step 3: Construct the optimal R*(A r ) of area A R by including pixels corresponding to Gj, G 2 , ... , 
Gar. , 

Step 4: Compute the SNR(A r ) obtained by using R*(A r ). If A R is greater than or equal to N, go to 
Step 5. Otherwise go to step 2. 

Step 5: Determine the largest SNR(A r ). This determines the optimal region of support. 

The above algorithm involves N sorts where each sort is of N values. Since A R changes by 
only 1 from one sort step to the next, we do not expect that the sorting order will change significantly 
from one step to the next. This can be used to speed up the algorithm still further. 

Ternary- Valued and Coupled Filters 

Optimal ternary- valued filters (with values -1, +1, and 0) in the presence input and detector noises 
have also been derived during the fellowship period. However due to lack of space, they are not 
discussed in this report. Similarly, the design of optimal coupled filters in the presence of input and 
detector noises have also been investigated. These results will be sent for publication in due course. 

SUMMARY 

In this report, the derivation of optimal real correlation filters when detector noise is not 
present is first presented. Then a model for the detector noise is presented. Using this model, the 
optimal filter design problem when detector noise is present is formulated. Derivations for the 
optimal filter when the filter is an unconstrained complex filter, real filter, and phase only filter are 
presented. Algorithms for their design are also presented. Implementation and testing of these 
algorithms will be taken up during the follow-up grant period. 


Of) - 


IS(/)I 


-r + p " (/) 

a r 


(55) 
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